Information processing apparatus and non-transitory computer readable medium

ABSTRACT

An information processing apparatus includes a processor configured to acquire from a storing unit a target image that is obtained by photographing a target and stored in the storing unit and a processing image that is stored in the storing unit so as to be consecutive to the target image and indicates an instruction for processing on the target image, and perform control such that the processing indicated by the processing image is performed on the target image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC 119 fromJapanese Patent Application No. 2020-092737 filed May 27, 2020.

BACKGROUND (i) Technical Field

The present disclosure relates to an information processing apparatusand a non-transitory computer readable medium.

(ii) Related Art

An image processing apparatus that additionally records a newlyphotographed moving image to a previously recorded moving image withoutrequiring a user to select a moving image to be additionally recordedevery time that the user performs photographing has been suggested (see,for example, Japanese Unexamined Patent Application Publication No.2015-171113).

The image processing apparatus described in Japanese Unexamined PatentApplication Publication No. 2015-171113 includes imaging means forimaging a subject to obtain an image, generating means for generating amoving image by the imaging means, specifying means for specifying acondition for additionally recording a newly generated second movingimage to a generated first moving image (for example, the samephotographing date), and recording means for additionally recording thesecond moving image to the first moving image in a case where the secondmoving image meets the specified condition and a first moving image thatmatches the condition is present.

SUMMARY

In the case where image processing provided by an external service isperformed for an image photographed with a mobile terminal or the like,an image photographed with the mobile terminal may be transmitted to theexternal service while the image and image processing to be performed onthe image being specified. However, it is troublesome for a user toassociate an image with processing to be performed on the image forevery image.

Aspects of non-limiting embodiments of the present disclosure relate toproviding an information processing apparatus and a non-transitorycomputer readable medium that are able to reduce, compared to a casewhere a user specifies a photographed image and image processing to beperformed on the image and issues an instruction to perform the imageprocessing, the number of operation times required for the user to issuean instruction to perform the image processing.

Aspects of certain non-limiting embodiments of the present disclosureaddress the above advantages and/or other advantages not describedabove. However, aspects of the non-limiting embodiments are not requiredto address the advantages described above, and aspects of thenon-limiting embodiments of the present disclosure may not addressadvantages described above.

According to an aspect of the present disclosure, there is provided aninformation processing apparatus including a processor configured toacquire from a storing unit a target image that is obtained byphotographing a target and stored in the storing unit and a processingimage that is stored in the storing unit so as to be consecutive to thetarget image and indicates an instruction for processing on the targetimage, and perform control such that the processing indicated by theprocessing image is performed on the target image.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the present disclosure will be described indetail based on the following figures, wherein:

FIG. 1 is a diagram illustrating an example of the configuration of aninformation processing system according to an exemplary embodiment ofthe present disclosure;

FIG. 2 is a block diagram illustrating an example of a control system ofan image forming apparatus;

FIG. 3 is a block diagram illustrating an example of a control system ofa personal computer;

FIG. 4 is a block diagram illustrating an example of a control system ofa mobile terminal;

FIG. 5 is a block diagram illustrating an example of a control system ofan information processing server;

FIG. 6 is a diagram illustrating an example of a setting table;

FIGS. 7A and 7B are diagrams illustrating examples of a managementtable;

FIG. 8 is a diagram illustrating an example of a setting screen;

FIGS. 9A to 9J are diagrams illustrating examples of photographingmethods using a mobile terminal;

FIGS. 10A to 10C are diagrams illustrating examples of photographingmethods using a mobile terminal;

FIG. 11 is a diagram illustrating an example of processing on aphotographed image uploaded to a storage server; and

FIG. 12 is a flowchart illustrating an example of an operation of aninformation processing server.

DETAILED DESCRIPTION

Hereinafter, exemplary embodiments of the present disclosure will bedescribed with reference to drawings. In the drawings, componentelements having substantially the same functions are referred to withthe same reference signs and redundant explanations will be omitted.

Summary of Exemplary Embodiment

An information processing apparatus according to an exemplary embodimentincludes a processor configured to acquire from a storing unit a targetimage that is obtained by photographing a target and stored in thestoring unit and a processing image that is stored in the storing unitso as to be consecutive to the target image and indicates an instructionfor processing on the target image, and perform control such that theprocessing indicated by the processing image is performed on the targetimage.

The “target” may be various objects such as an object represented onpaper (for example, a paper document, a photo book, or an image), anobject represented on a screen (for example, a screenshot of a Web pageor the like), and an object represented three-dimensionally (forexample, a human being, a building, or a vehicle). In an exemplaryembodiment, for example, a paper document is used as a target. “Beingstored in the storing unit so as to be consecutive to the target image”represents a state in which a processing image is stored in the storingunit such that the processing image is temporally before or after thetarget image. For example, a case where a processing image, a targetimage, and a processing image are stored in the storing unit in thatorder, a case where a processing image and a target image are stored inthe storing unit in that order, a case where a target image and aprocessing image are stored in the storing unit in that order, andsimilar cases correspond to the state mentioned above.

Exemplary Embodiment

FIG. 1 is a diagram illustrating an example of the configuration of aninformation processing system according to an exemplary embodiment ofthe present disclosure. An information processing system 1 includes animage forming apparatus 2, a personal computer (hereinafter, referred toas a PC) 3, a mobile terminal 4, an internal network 5 to which theimage forming apparatus 2 and the PC 3 are connected, and an informationprocessing server 7 and a storage server 8 that are connected to theinternal network 5 via an external network 6. In FIG. 1, one PC 3 andone mobile terminal 4 are illustrated. However, two or more PCs 3 andtwo or more mobile terminals 4 may be provided. The informationprocessing server 7 is an example of an information processingapparatus. The storage server 8 is an example of a storing unit.

The image forming apparatus 2 is a multifunction machine having aplurality of functions including, for example, a scan function, a printfunction, a copy function an electronic mail function, and a facsimiletransmission and reception function.

The mobile terminal 4 is, for example, a tablet terminal, amultifunction telephone set (may be called a smartphone), or the likethat has a camera function. The mobile terminal 4 communicates with theinformation processing server 7 and the storage server 8 via theexternal network 6.

The internal network 5 is, for example, a communication network such asa local area network (LAN) or a wide area network (WAN) and may be wiredor wireless. The external network 6 may be a public telephone network,the Internet, or the like.

The information processing server 7 and the storage server 8 may beconnected by an intranet or the like and may be configured as a physicalsingle server. The information processing server 7 and the storageserver 8 may be an example of an information processing apparatus.

The information processing server 7 provides a document processingservice. In the document processing service, processing (for example,server storage, e-mail transmission, a PDF conversion, or FAXtransmission) selected for an image or a document (may be called adocument file) uploaded from the image forming apparatus 2, the PC 3, orthe mobile terminal 4 is performed.

For use of the document processing service using the image formingapparatus 2, setting for linkage with the information processing server7 is performed, a Web screen is displayed on an operation display unit22 of the image forming apparatus 2 (see FIG. 2), processing is selectedfrom a menu displayed on the Web screen, and a paper document is scannedwith an image reading unit 23 (see FIG. 2). Accordingly, the documentprocessing service is performed for the image obtained by scanning (maybe called a scanned image). In the case where the paper documentcontains a plurality of pages, a plurality of scanned images areprocessed as a set of electronic files (for example, stored in afolder).

For use of the document processing service using the PC 3, setting forlinkage with the information processing server 7 is performed, a Webscreen is displayed on a display unit 33 of the PC 3 (see FIG. 3), andan input unit (see FIG. 3) is operated such that a document file isdragged and dropped on an image indicating processing displayed on theWeb screen. Accordingly, the document processing service is performed.

For use of the document processing service using the mobile terminal 4,setting for linkage with the information processing server 7 and thestorage server 8 is performed, and data owned by a user is sharedbetween the mobile terminal 4 and a service provided by the storageserver 8. Accordingly, the document processing service is performed.Details of the case where the document processing service is used usingthe mobile terminal 4 will be described later. Furthermore, the mobileterminal 4 includes, as an operating system (OS) or application, afunction for automatically or manually backing up photographs taken bythe mobile terminal 4 on the storage server 8, by being linked with thestorage server 8.

The storage server 8 provides a storage service that shares a file oncloud and includes a database (DB) 80 in which files for each user areable to be stored. A management table 81 (see FIGS. 7A and 7B) for eachuser is stored in the DB 80. For example, image data captured by themobile terminal 4 and backed up is stored in the management table 81.

(Configuration of Image Forming Apparatus)

FIG. 2 is a block diagram illustrating an example of a control system ofthe image forming apparatus 2. The image forming apparatus 2 includes acontroller 20 that controls units of the image forming apparatus 2. Astoring unit 21, an operation display unit 22, an image reading unit 23,an image forming unit 24, and a communication unit 25 are connected tothe controller 20.

The controller 20 includes a processor such as a central processing unit(CPU), an interface, and the like. The processor controls the units ofthe image forming apparatus 2 by executing a program stored in thestoring unit 21.

The storing unit 21 includes a read only memory (ROM), a random accessmemory (RAM), a hard disk, and the like. A program and various types ofinformation such as a document file are stored in the storing unit 21.The program includes a Web browser for browsing a Web page.

The operation display unit 22 receives and displays information. Theoperation display unit 22 is, for example, a touch panel display. Theoperation display unit 22 has a configuration in which a touch panel issuperimposed on a display such as a liquid crystal display.

The image reading unit 23 includes an automatic paper feeder provided ona platen and a scanner that optically reads a document image from adocument placed on the platen or a document transmitted from theautomatic paper feeder.

The image forming unit 24 creates a printed material by forming, forexample, using an electrophotographic system or an inkjet system, animage on paper fed from a paper feed cassette.

The communication unit 25 communicates with the PC 3 via the internalnetwork 5 and communicates with an external apparatus such as theinformation processing server 7 and the storage server 8 via theinternal network 5 and the external network 6.

(Configuration of PC)

FIG. 3 is a block diagram illustrating an example of a control system ofthe PC 3. The PC 3 includes a controller 30 that controls units of thePC 3, a storing unit 31 that stores a program including a Web blower andvarious types of information such as data, an input unit 32 that isimplemented by a keyboard, a mouse, and the like, a display unit 33 thatis implemented by a liquid crystal display or the like, and acommunication unit 34 that is connected to the internal network 5.

(Configuration of Mobile Terminal)

FIG. 4 is a block diagram illustrating an example of a control system ofthe mobile terminal 4. The mobile terminal 4 includes a controller 40that controls units of the mobile terminal 4, a storing unit 41 thatstores a program including a Web browser and various types ofinformation such as data, an operation display unit 42 that isimplemented by a touch panel display or the like, a camera 43, and acommunication unit 44 that is connected to the external network 6.

The mobile terminal 4 has a posture detection function implemented by agravity sensor or the like. In the case where the mobile terminal 4 in avertical position posture performs photographing using the camera 43, avertically long photographed image (for example,vertical:horizontal=4:3) is obtained. In the case where the mobileterminal 4 in a horizontal position posture performs photographing usingthe camera 43, a horizontally long photographed image (for example,horizontal:vertical=4:3) is obtained.

An example of the case where the document processing service is usedusing the mobile terminal 4 will be described. A user takes a photographas one of first processing images 430 a to 430 d (see FIGS. 9A to 9H)while covering part of a lens of the camera 43 of the mobile terminal 4(for example, a left half, a right half, a top half, or a bottom half),and then photographs one or more paper documents as a target image(s)431 (see FIG. 10B). After that, the user takes a photograph as a secondprocessing image 430 e (see FIGS. 9I and 9J) while covering the entirelens of the camera 43, and then uploads the corresponding one of thefirst processing images 430 a to 430 d, the target image(s) 431, and thesecond processing image 430 e (hereinafter, these images may be referredto as “photographed images”) to the storage server 8. Accordingly, thedocument processing service is performed.

That is, the document processing service is performed when theinformation processing server 7 acquires the target image(s) 431 thatis/are sandwiched between the corresponding one of the first processingimages 430 a to 430 d and the second processing image 430 e from thestorage server 8 and performs content of processing indicated by thecorresponding one of the first processing images 430 a to 430 d for thetarget image 431. Meaning of the first processing images 430 a to 430 dand the second processing image 430 e will be described later.

(Configuration of Information Processing Server)

FIG. 5 is a block diagram illustrating an example of a control system ofthe information processing server 7. The information processing server 7includes a controller 70 that controls units of the informationprocessing server 7, a storing unit 71 that stores a program and varioustypes of information such as data, an input unit 72 that is implementedby a keyboard, a mouse, and the like, a display unit 73 that isimplemented by a liquid crystal display or the like, an image processingunit 74 that performs image processing such as processing for convertinga format of image data, and a communication unit 75 that is connected tothe external network 6.

The controller 70 includes a processor 70 a such as a CPU, an interface,and the like. The processor 70 a functions as setting means 700,acquiring means 701, execution control means 702, and the like whenexecuting a program 710 stored in the storing unit 71. The details ofthe setting means 700, the acquiring means 701, and the executioncontrol means 702 will be described later.

The storing unit 71 includes a ROM, a RAM, a hard disk, and the like.The program 710, a setting table 711 (see FIG. 6), and various types ofinformation such as an electronic file 712 are stored in the storingunit 71.

The image processing unit 74 performs, for example, processing forconverting a photographed image or a word document in an RGB format intoimage data in a portable document format (PDF) format.

The communication unit 75 transmits and receives information to and fromthe mobile terminal 4 via the external network 6, and transmits andreceives information to and from the image forming apparatus 2 and thePC 3 via the external network 6 and the internal network 5.

FIG. 6 is a diagram illustrating an example of the setting table 711.The setting table 711 contains a plurality of items such as “imagepattern ID”, “image pattern”, and “instruction for processing”. “Imagepattern ID” is used for identifying an image pattern. Images captured bythe mobile terminal 4 are registered as image patterns 423 a to 423 e in“Image pattern”. An instruction for processing includes content of theprocessing and execution of the processing. In “instruction forprocessing”, content of processing to be performed under control of theinformation processing server 7 and execution of the processing areregistered. An image pattern is an example of characteristics of aprocessing image.

FIGS. 7A and 7B are diagrams illustrating examples of the managementtable 81 stored in the DB 80 of the storage server 8. The managementtable 81 contains a plurality of items such as “date and time”, “imagedata”, “image pattern ID”, and “confirmation flag”.

Dates and times when image data were uploaded to the storage server 8(for example, year, month, date, hour, minute, and second) are recordedin “date and time” by the storage server 8 in the order in which theimage data were uploaded.

Image data such as the processing images 430 and the target images 431(hereinafter, may be referred to as “photographed images”) are recordedin “image data” by the storage server 8 in the order in which theprocessing images 430 and the target images 431 were uploaded.

“Image pattern ID” corresponds to “image pattern ID” in FIG. 6 and isrecorded by the acquiring means 701 of the information processing server7. That is, in the case where a photographed image corresponds to anyone of the image patterns illustrated in FIG. 6, a corresponding imagepattern ID (01, 02, 03, 04, or 05) is recorded, and in the case where aphotographed image corresponds to none of the image patterns illustratedin FIG. 6, “xx” is recorded as an image pattern ID.

A value “0” is recorded as a default value in “confirmation flag”. Afterrecording an image pattern ID in “image pattern ID”, the acquiring means701 records “1” in “confirmation flag”.

Next, details of the setting means 700, the acquiring means 701, and theexecution control means 702 of the controller 70 will be described.

The setting means 700 receives registration of image patterns of thefirst processing images 430 a, 430 b, 430 c, and 430 d and the secondprocessing image 430 e (hereinafter, may be referred to as “processingimages 430” when these processing images are collectively referred to)from a user.

Specifically, the setting means 700 converts the processing images 430uploaded to a setting screen 420 into the image patterns 423 a to 423 d(hereinafter, may be referred to as “image patterns 423” when theseimage patterns are collectively referred to) using a known patternmatching method. In the pattern matching method, for example, aplurality of image patterns with black regions of different shapes areprepared, and an image pattern with the highest similarity with aprocessing image 430 is selected. In the case where a photographed imagehas a rectangular shape, an image pattern may be a square shape or arectangular shape corresponding to the shape of the photographed image.The pattern matching method is not necessarily used. Conversion into animage pattern may not be performed. For example, characteristics of animage registered in advance by a user and content of processing may bestored in association with each other. In the case where the similaritybetween characteristics of a processing image and the characteristics ofthe image registered in advance is equal to or more than a threshold,the processing image is determined to correspond to the image registeredin advance.

Furthermore, the setting means 700 records content of processingassociated with a converted image pattern 423 into the setting table711. For example, content of a plurality of pieces of processing isprepared in advance, and a user may select content of processing to beassociated with the image pattern 423.

The acquiring means 701 acquires from the storage server 8 the targetimage 431 obtained by photographing a target and stored in the storageserver 8 and the processing images 430 stored in the storage server 8 soas to be consecutive to the target image 431 and indicating aninstruction for processing on the target image 431. Specifically, forexample, the acquiring means 701 acquires a target image 431 between anunconfirmed first processing image 430 a with an image pattern ID of 01and an unconfirmed second processing image 430 e with an image patternID of 05 from the management table 81 of the storage server 8, andstores the acquired target image 431 as the electronic file 712 into thestoring unit 71.

The first processing image 430 a may indicate a starting position ofprocessing, and the second processing image 430 e may indicate an endingposition of the processing. The first processing image 430 a mayindicate content of processing, and the second processing image 430 emay indicate an instruction for execution of the processing.Furthermore, the first processing image 430 a may indicate a startingposition of processing and content of the processing, and the secondprocessing image 430 e may indicate an ending position of the processingand an instruction for execution of the processing.

The first processing image 430 a may not indicate content of processing,and the second processing image 430 e may indicate the content of theprocessing and execution of the processing. Furthermore, in the casewhere an interval between photographing times of target images 431 isequal to or longer than a predetermined time (for example, one minute),only target images 431 captured before the interval that is equal to orlonger than the predetermined time may be acquired as targets forprocessing. In this case, the last target image 431 as a target forprocessing may also serve as the second processing image 430 e.

In the case where no processing image 430 is acquired, the acquiringmeans 701 may perform control such that a warning that prompts the userto add a processing image 430 is provided.

The execution control means 702 controls the units of the informationprocessing server 7 and the storage server 8 to perform processingindicated by the first processing images 430 a to 430 d for the targetimage 431 in accordance with an instruction for executing the processingindicated by the second processing image 430 e.

Specifically, the execution control means 702 performs control such thatcontent of processing corresponding to an image pattern ID of one of thefirst processing images 430 a to 430 d determined by the acquiring means701, with reference to the setting table 711 in FIG. 6, is performed forthe target image 431 stored in the storing unit 71.

That is, in the case where the image pattern ID is 01 and an instructionfor processing is server storage, the execution control means 702controls a server, for example, the DB 80 of the storage server 8, tostore the target image 431. In the case where the image pattern ID is 02and an instruction for processing is e-mail transmission, the executioncontrol means 702 controls the communication unit 75 to transmit thetarget image 431 to a terminal with a specified e-mail address (forexample, the PC 3). In the case where the image pattern ID is 03 and aninstruction for processing is PDF conversion, the execution controlmeans 702 controls the image processing unit 74 to convert the targetimage 431 into the PDF format. In the case where the image pattern ID is04 and an instruction for processing is FAX transmission, the executioncontrol means 702 controls the communication unit 75 to transmit thetarget image 431 to a transmission destination with a specified FAXnumber.

Because the first processing images 430 a to 430 d are stored prior totarget images 431 into the storage server 8, the execution control means702 may start processing corresponding to the first processing images430 a to 430 d for target images 431 sequentially, instead of waitinguntil all the target images 431 have been acquired and then performingprocessing.

(Operation of Information Processing System)

Next, an example of an operation of the information processing system 1will be described with reference to FIGS. 8 to 12. FIG. 12 is aflowchart illustrating an example of an operation of the informationprocessing server 7.

(1) Setting of Image Pattern

A user operates the mobile terminal 4 to access the informationprocessing server 7. The setting means 700 of the information processingserver 7 transmits information of a setting screen using a Web page tothe mobile terminal 4. The controller 40 of the mobile terminal 4 causesthe setting screen to be displayed on the operation display unit 42.

FIG. 8 is a diagram illustrating an example of a setting screen. In FIG.8, a state after setting is completed is illustrated. The setting screen420 contains a storage selecting unit 421 for selecting a storageservice to be linked with the mobile terminal 4, a check box 422 forselecting whether or not to allow access to the storage server 8, andprocessing content input fields 424 a to 424 e (hereinafter, may bereferred to as “processing content input fields 424” in the case wherethe processing content input fields 424 a to 424 e are collectivelyreferred to) into which content of processing and execution of theprocessing are input for the image patterns 423 a to 423 e. Content ofprocessing and execution of the processing are input to the processingcontent input fields 424 by performing selection using a pull-down menu.

In the example illustrated in FIG. 8, a case where “server storage” isinput to the processing content input field 424 a for the image pattern423 a whose right part is black, “e-mail transmission” is input to theprocessing content input field 424 b for the image pattern 423 b whoseleft part is black, “PDF conversion” is input to the processing contentinput field 424 c for the image pattern 423 c whose top part is black,“FAX transmission” is input to the processing content input field 424 dfor the image pattern 423 d whose bottom part is black, and “processingexecution” is input to the processing content input field 424 e for theimage pattern 423 e with the entire black region is illustrated.

An instruction for processing will now be explained. “Server storage”represents, as content of processing, processing for storing aphotographed image into a server specified on a details setting screen(for example, the storage server 8). “E-mail transmission” represents,as content of processing, processing for transmitting a photographedimage to an e-mail address specified on the details setting screen. “PDFconversion” represents, as content of processing, processing forconverting a photographed image into the PDF format. “FAX transmission”represents, as content of processing, processing for transmitting, byfacsimile, a photographed image to a transmission destination specifiedon the details setting screen. “Processing execution” representsexecution of processing. Specifically, “processing execution” representsexecution of processing such as server storage, e-mail transmission, PDFconversion, and FAX transmission. Processing is not limited to theprocessing mentioned above. Processing typically performed for an image,such as rotation of an image, inversion of an image, color adjustment(color interpolation processing, color correction, etc.) of an image,resolution adjustment, and contour enhancement processing, may also beincluded.

When photographing is performed while a left part of the lens of thecamera 43 of the mobile terminal 4 being covered as illustrated in FIG.9A, the first processing image 430 a illustrated in FIG. 9B is obtained.When photographing is performed while a right part of the lens of thecamera 43 of the mobile terminal 4 being covered as illustrated in FIG.9C, the first processing image 430 b illustrated in FIG. 9D is obtained.When photographing is performed while a top part of the lens of thecamera 43 of the mobile terminal 4 being covered as illustrated in FIG.9E, the first processing image 430 c illustrated in FIG. 9F is obtained.When photographing is performed while a bottom part of the lens of thecamera 43 of the mobile terminal 4 being covered as illustrated in FIG.9G, the first processing image 430 d illustrated in FIG. 9H is obtained.When photographing is performed while the entire lens of the camera 43of the mobile terminal 4 being covered as illustrated in FIG. 9I, thesecond processing image 430 e illustrated in FIG. 9J is obtained.Accordingly, when the user performs photographing sequentially, forexample, while the left part, the right part, the top part, the bottompart, and the entire lens of the camera 43 being covered, the firstprocessing images 430 a to 430 d and the second processing image 430 eare stored in the storing unit 41. The controller 40 of the mobileterminal 4 transmits a plurality of (for example, five) processingimages 430 stored in the storing unit 41 to the information processingserver 7 via the external network 6.

The setting means 700 of the information processing server 7 convertsthe uploaded processing images 430 into the image patterns 423.

The setting means 700 sequentially arranges the image patterns 423 a to423 d in a photographed order on the setting screen 420. The useroperates the processing content input fields 424 to select content ofprocessing and execution of the processing using the pull-down menu.“Server storage”, “e-mail transmission”, “PDF conversion”, and “FAXtransmission” are prepared in advance as the content of processing.

When instructions for the processing (the content of the processing andexecution of the processing) are selected for the image patterns 423 ato 423 e as described above, the setting means 700 provides the imagepatterns with image pattern IDs and registers the image pattern IDsalong with the instructions for the processing into the setting table711, as illustrated in FIG. 6.

The first processing images 430 a to 430 d in FIGS. 9B, 9D, 9F, and 9Hcorrespond to the image pattern IDs 01 to 04 and the image patterns 423a to 423 d in FIG. 6. The second processing image 430 e in FIG. 9Jcorresponds to the image pattern ID xx and the image pattern 423 e inFIG. 6.

(2) Uploading of Image Obtained by Photographing Paper Document

Next, a case where an image obtained by photographing a paper documentby a user using the mobile terminal 4 is uploaded to the storage server8 will be explained. A case where “server storage” is selected ascontent of processing will be explained.

FIGS. 10A to 10C are diagrams for explaining examples of methods foruploading images obtained by photographing using the mobile terminal 4.First, a user performs photographing while covering a left part of thelens of the camera 43, and obtains the first processing image 430 a, asillustrated in FIG. 10A. The controller 40 of the mobile terminal 4causes the obtained first processing image 430 a to be stored into thestoring unit 41, and uploads the first processing image 430 a to thestorage server 8, which is linked with the mobile terminal 4, via theexternal network 6.

Next, the user performs photographing while directing the camera 43 ofthe mobile terminal 4 toward a paper document that is desired to beconverted into an electronic file. For example, n images are captured inthe page order. The controller 40 of the mobile terminal 4 causes theobtained n target images 431 to be sequentially stored into the storingunit 41, and sequentially uploads the n target images 431 to the storageserver 8.

Finally, the user performs photographing while covering the entire lensof the camera 43 to obtain the second processing image 430 e. Thecontroller 40 of the mobile terminal 4 causes the obtained secondprocessing image 430 e to be stored into the storing unit 41, anduploads the second processing image 430 e to the storage server 8.

In the case where an operation for uploading is performed, thecontroller 40 of the mobile terminal 4 may upload the first processingimage 430 a, the target image 431, and the second processing image 430 estored in the storing unit 41 to the storage server 8. The controller 40of the mobile terminal 4 may upload the first processing image 430 a,the target image 431, and the second processing image 430 e to thestorage server 8 without causing them to be stored in the storing unit41.

The storage server 8 records the first processing image 430 a, thetarget image 431, and the second processing image 430 e uploaded fromthe mobile terminal 4, along with uploaded date and time, into themanagement table 81, as illustrated in FIG. 7A. In FIG. 7A, “1” isrecorded in “confirmation flag” in the first and second rows. Thisindicates that confirmation by the acquiring means 701 is completed forthe first and second rows.

(3) Processing for Uploaded Images

Processing for uploaded images will be explained with reference to aflowchart of FIG. 12.

The acquiring means 701 of the information processing server 7 extractsadded image data from the storage server 8 in chronological order (S1).That is, the acquiring means 701 sequentially extracts image data withthe confirmation flag “0” in the order from the latest to the oldestfrom the management table 81 of the storage server 8.

The acquiring means 701 determines whether or not there is anyunconfirmed image (S2). In the case where image data with theconfirmation flag “0” in the management table 81 is recorded, it isdetermined that there is an unconfirmed image. In the case illustratedin FIG. 7A, image data with the uploaded date and time “2020-03-2509:23:46” and image data updated at later times have not been confirmed.

In the case where there is unconfirmed image data (S2: Yes), theacquiring means 701 determines whether or not the unconfirmed image datacorresponds to one of the first processing images 430 a to 430 d (S3).That is, the acquiring means 701 determines, using a pattern matchingmethod similar to that used by the setting means 700, whether or not theunconfirmed image data corresponds to one of the image patterns with theimage pattern IDs 01 to 04 illustrated in FIG. 6.

In the case where it is determined that the unconfirmed image datacorresponds to one of the first processing images 430 a to 430 d (S3:Yes), the acquiring means 701 records “01” as “image pattern ID” andrecords “1” as “confirmation flag”, as illustrated in FIG. 7B.

The acquiring means 701 determines whether or not the next image data isunconfirmed image data (S4). In the case where image data with theconfirmation flag “0” in the management table 81 is recorded, the imagedata is determined to be unconfirmed image data.

In the case where the next image data is unconfirmed image data (S4:Yes), the acquiring means 701 determines whether or not an imagecorresponding to the image data is the second processing image 430 e(S5). That is, the acquiring means 701 determines, using the patternmatching method similar to that used by the setting means 700, whetheror not the unconfirmed image data corresponds to the image pattern withthe image pattern ID 05 illustrated in FIG. 6.

In the case where the image corresponding to the unconfirmed image datais not the second processing image 430 e (S5: No), the acquiring means701 repeats the processing (S6) for determining whether or not the nextimage data is unconfirmed image data and the processing (S7) fordetermining whether or not the image corresponding to the unconfirmedimage data is the second processing image 430 e until it is determinedthat the unconfirmed image is the second processing image 430 e.

In the case where it is determined that the unconfirmed image is thesecond processing image 430 e (S7: Yes), the acquiring means 701acquires, as the target image 431, image data between one of the firstprocessing images 430 a to 430 d (in FIG. 11, the first processing image430 a) and the second processing image 430 e, and causes the targetimage 431 to be stored into the storing unit 71 of the informationprocessing server 7, as illustrated in FIG. 11 (S8).

The execution control means 702 performs control such that content ofprocessing corresponding to the image pattern ID of one of the firstprocessing images 430 a to 430 d determined in step S3 is performed forthe target image 431 stored in the storing unit 71, as illustrated inFIG. 11 (S9). In the case illustrated in FIG. 11, the execution controlmeans 702 controls the communication unit 75 to cause the target images431 to be stored as a set of electronic files into the storage server 8.

In the case where it is determined in step S2 that there is nounconfirmed image (S2: No), the acquiring means 701 controls the mobileterminal 4 to display a notification indicating that the process hasended (S10).

In the case where it is determined in step S3 that the unconfirmed imageis none of the first processing images 430 a to 430 d (S3: No), theacquiring means 701 determines whether or not the unconfirmed image isthe second processing image 430 e (S11).

In the case where it is determined in step S4 that the next image is notan unconfirmed image (S4: No), in the case where it is determined instep S5 that the next image is the second processing image 430 e (S5:Yes), in the case where it is determined in step S6 that the next imageis not an unconfirmed image (S6: No), or it is determined in step S11that the unconfirmed image is the second processing image 430 e (S11:Yes), the acquiring means 701 controls the mobile terminal 4 to provideerror display (S12). As error display, display corresponding to contentof error, for example, in the case where the target image 431, the firstprocessing images 430 a to 430 d, or the second processing image 430 eis missing, a warning that prompts the user to add the missing image, isprovided. Error display is an example of a warning. In the case wherethe first processing images 430 a to 430 d are missing, the user may besimply asked about the content of the processing. For example, a userinterface (UI) screen indicating “Please select processing you wish toperform for the target images.” as well as thumbnail images for theacquired target images 431 may be displayed. Thus, the acquiring means701 may receive the content of the processing.

With the configuration described above, there is no need to perform anoperation for displaying a menu for processing for the target images431. Thus, compared to a configuration that requires displaying of amenu for processing, the number of operation times may be reduced.

First Modification

In the exemplary embodiment described above, the first processing images430 a to 430 d and the second processing image 430 e are captured beforeand after the target image 431 is captured and are uploaded to thestorage server 8. However, the processing images 430 may be captured inadvance and may be stored as the image patterns 423 a to 423 e into thestoring unit 41 of the mobile terminal 4. After selecting the imagepatterns 423 a to 423 d corresponding to the first processing images 430a to 430 d, a user may capture the target images 431 and then select theimage pattern 423 e corresponding to the second processing image 430 e.In this case, the controller 40 of the mobile terminal 4 uploads theimage pattern IDs and the target image 431 to the storage server 8,instead of transmitting the processing images 430.

Second Modification

In the exemplary embodiment described above, the processing images 430are identified according to image pattern. However, the processingimages 430 may be identified according to color or combination of imagepattern and color. Color or a combination of color and image pattern isan example of characteristics of a processing image.

Third Modification

In the exemplary embodiment described above, after one of the firstprocessing images 430 a to 430 d is captured, the target image 431 iscaptured. However, two or more of the first processing images 430 a to430 d are captured in a consecutive manner, the target image 431 may becaptured.

In this case, the execution control means 702 of the informationprocessing server 7 performs control such that content of processingindicated by the consecutive two or more of first processing images 430a to 430 d is performed for the target image 431 in a consecutivemanner.

For example, after the first processing image 430 c with the content ofprocessing “PDF conversion” (the image pattern ID is “03”) is captured,the first processing image 430 b with the content of processing “e-mailtransmission” (the image pattern ID is “02”) is captured. Subsequently,the target image 431 is captured, and the second processing image 430 eis captured. Then, the first processing image 430 c, the firstprocessing image 430 b, the target image 431, and the second processingimage 430 e are uploaded to the storage server 8. In this case, thetarget image 431 is converted into the PDF format and transmitted to aspecified e-mail address by the information processing server 7.

Fourth Modification

A separation image for separating between target images 431 may beinserted between the target images 431. For example, the firstprocessing images 430 a to 430 d, a target image 431, a target image431, . . . , a separation image, a target image 431, a target image 431,. . . , and the second processing image 430 e may be captured in thatorder and uploaded to the storage server 8.

In this case, the execution control means 702 performs control such thatprocessing indicated by the first processing images 430 a to 430 d isperformed for individual target image sets separated by a separationimage. Each processed target image set separated by a separation imageis stored as a set of electronic files (for example, stored into afolder).

Fifth Modification

The information processing server 7 and the storage server 8 may beprovided in the internal network 5. In this case, the mobile terminal 4communicates with the information processing server 7 and the storageserver 8 via the internal network 5 in a wireless manner.

Exemplary embodiments of the present disclosure have been describedabove. However, the exemplary embodiments of the present disclosure arenot limited to the exemplary embodiments described above and variousmodifications and implementations may be made to the present disclosure.

The means of a processor may be partially or entirely configured as ahardware circuit such as a field programmable gate array (FPGA) or anapplication specific integrated circuit (ASIC).

Furthermore, part of components in the exemplary embodiments describedabove may be omitted or changed. In the flow of a process in theexemplary embodiments described above, addition, deletion, change,exchange, and the like of steps may be performed. A program used in theexemplary embodiments described above may be recorded in acomputer-readable recording medium such as a compact disc-read onlymemory (CD-ROM) and provided. The program may be stored in an externalserver such as a cloud server and used via a network.

In the embodiments above, the term “processor” refers to hardware in abroad sense. Examples of the processor include general processors (e.g.,CPU: Central Processing Unit) and dedicated processors (e.g., GPU:Graphics Processing Unit, ASIC: Application Specific Integrated Circuit,FPGA: Field Programmable Gate Array, and programmable logic device).

In the embodiments above, the term “processor” is broad enough toencompass one processor or plural processors in collaboration which arelocated physically apart from each other but may work cooperatively. Theorder of operations of the processor is not limited to one described inthe embodiments above, and may be changed.

The foregoing description of the exemplary embodiments of the presentdisclosure has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit thedisclosure to the precise forms disclosed. Obviously, many modificationsand variations will be apparent to practitioners skilled in the art. Theembodiments were chosen and described in order to best explain theprinciples of the disclosure and its practical applications, therebyenabling others skilled in the art to understand the disclosure forvarious embodiments and with the various modifications as are suited tothe particular use contemplated. It is intended that the scope of thedisclosure be defined by the following claims and their equivalents.

What is claimed is:
 1. An information processing apparatus comprising: aprocessor configured to acquire from a storing unit a target image thatis obtained by photographing a target and stored in the storing unit anda processing image that is stored in the storing unit so as to beconsecutive to the target image and indicates an instruction forprocessing on the target image, and perform control such that theprocessing indicated by the processing image is performed on the targetimage.
 2. The information processing apparatus according to claim 1,wherein the processing image includes a first processing imageindicating a starting position of the processing and a second processingimage indicating an ending position of the processing.
 3. Theinformation processing apparatus according to claim 2, wherein the firstprocessing image is stored prior to the target image into the storingunit.
 4. The information processing apparatus according to claim 1,wherein the processing image includes a first processing imageindicating content of the processing and a second processing imageindicating an instruction for execution of the processing on the targetimage.
 5. The information processing apparatus according to claim 2,wherein the processing image includes a first processing imageindicating content of the processing and a second processing imageindicating an instruction for execution of the processing on the targetimage.
 6. The information processing apparatus according to claim 4,wherein the processor is configured to, in a case where a plurality offirst processing images are acquired in a consecutive manner, performcontrol such that the content of the processing indicated by theacquired first processing images is performed on the target image in aconsecutive manner.
 7. The information processing apparatus according toclaim 5, wherein the processor is configured to, in a case where aplurality of first processing images are acquired in a consecutivemanner, perform control such that the content of the processingindicated by the acquired first processing images is performed on thetarget image in a consecutive manner.
 8. The information processingapparatus according to claim 1, wherein the processing image is selectedfrom a photographed image or information registered in advance.
 9. Theinformation processing apparatus according to claim 2, wherein theprocessing image is selected from a photographed image or informationregistered in advance.
 10. The information processing apparatusaccording to claim 3, wherein the processing image is selected from aphotographed image or information registered in advance.
 11. Theinformation processing apparatus according to claim 4, wherein theprocessing image is selected from a photographed image or informationregistered in advance.
 12. The information processing apparatusaccording to claim 5, wherein the processing image is selected from aphotographed image or information registered in advance.
 13. Theinformation processing apparatus according to claim 6, wherein theprocessing image is selected from a photographed image or informationregistered in advance.
 14. The information processing apparatusaccording to claim 7, wherein the processing image is selected from aphotographed image or information registered in advance.
 15. Theinformation processing apparatus according to claim 1, wherein in a casewhere an interval between photographing times of target images is equalto or longer than a predetermined time, only target images capturedbefore the interval that is equal to or longer than the predeterminedtime are used as targets for the processing.
 16. The informationprocessing apparatus according to claim 2, wherein in a case where aninterval between photographing times of target images is equal to orlonger than a predetermined time, only target images captured before theinterval that is equal to or longer than the predetermined time are usedas targets for the processing.
 17. The information processing apparatusaccording to claim 1, wherein the processor is configured to, in a casewhere no processing image is acquired, perform control such that awarning is displayed.
 18. The information processing apparatus accordingto claim 1, wherein the processor is configured to receive registrationof characteristics of the processing image from a user.
 19. Theinformation processing apparatus according to claim 1, wherein theprocessor is configured to acquire a separation image for separatingbetween target images from the storing unit and perform control suchthat the processing is performed for individual target image setsseparated by the separation image.
 20. A non-transitory computerreadable medium storing a program causing a computer to execute aprocess for information processing, the process comprising: acquiringfrom a storing unit a target image that is obtained by photographing atarget and stored in the storing unit and a processing image that isstored in the storing unit so as to be consecutive to the target imageand indicates an instruction for processing on the target image, andperforming control such that the processing indicated by the processingimage is performed on the target image.